Mobile terminal and method for operating the same

ABSTRACT

A method of operating a mobile terminal includes obtaining at least one image and determining event information that is to be associated with the obtained at least one image. The method also includes storing, in computer memory, the obtained at least one image and information that associates the obtained at least one image with the event information. The method additionally includes detecting an event on the mobile terminal, and determining that the detected event corresponds to the event information. The method further includes displaying, on a display of the mobile terminal and based on the determination that the detected event corresponds to the event information, a first image among the at least one image that has been stored and associated with the event information.

CROSS-REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit ofan earlier filing date and right of priority to Korean PatentApplication No. 10-2015-0126176, filed on Sep. 7, 2015 in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein by reference in its entirety.

TECHNICAL FIELD

The present application relates to a mobile terminal configured toclassify, manage, and display images.

BACKGROUND

With the development of mobile communication technology, the utilizationof mobile terminals has continued to increase. Mobile terminals, such asmobile phones and other portable electronic devices, are being used fora variety of purposes. For example, mobile terminals typically includeimage capture functionality, such as a camera, that allows users tocapture images of events during the course of their everyday lives.

SUMMARY

Systems and techniques described herein enable a mobile terminal toclassify, manage, and display images based on event information that isassociated with the images. In some implementations, the mobile terminalmay be configured to detect events that occur in a user's environment,and display previously-stored images that have been associated with thedetect events.

In one aspect, a method of operating a mobile terminal may includeobtaining at least one image; determining event information that is tobe associated with the obtained at least one image; storing, in computermemory, the obtained at least one image and information that associatesthe obtained at least one image with the event information; detecting anevent on the mobile terminal; determining that the detected eventcorresponds to the event information; and displaying, on a display ofthe mobile terminal and based on the determination that the detectedevent corresponds to the event information, a first image among the atleast one image that has been stored and associated with the eventinformation.

In some implementations, obtaining the at least one image may includeobtaining, by the mobile terminal, an image that was captured through afirst image-capturing mode of the mobile terminal. Determining the eventinformation that is to be associated with the obtained at least oneimage may include determining that a user input has been received; anddetermining, based on the user input, the event information as being atleast one of time information indicating a time at which the at leastone image is to be displayed, location information indicating a locationwhere the at least one image is to be displayed, or applicationinformation indicating an application that is to be displayed togetherwith the at least one image.

In some implementations, obtaining the at least one image that wascaptured through the first image-capturing mode of the mobile terminalmay include obtaining an image that was captured while a screen of themobile terminal was turned off.

In some implementations, the method may further include displaying, onthe display of the mobile terminal, a plurality of images including thefirst image. Determining the event information that is to be associatedwith the obtained at least one image may include: determining that auser input has been received indicating the event information that is tobe associated with the first image of the plurality of images; anddetermining, based on the user input, the event information as includingat least one of time information indicating a time at which the firstimage is to be displayed, location information indicating a locationwhere the first image is to be displayed, and application informationindicating an application that is to be displayed together with thefirst image.

In some implementations, obtaining the at least one image may includeobtaining, by the mobile terminal, an image that was captured through afirst image-capturing mode of the mobile terminal. Determining the eventinformation that is to be associated with the at least one image mayinclude: determining the event information from the image that wascaptured through the first image-capturing mode.

In some implementations, determining the event information from theimage that was captured through the first image-capturing mode mayinclude: determining the event information based on at least one of anobject detected in the image, a text detected in the image, locationinformation indicating a location where the image was captured, timeinformation indicating a time at which the image was captured, orapplication information indicating an application that was executed whenthe image was captured.

In some implementations, displaying, on the display of the mobileterminal and based on the determination that the detected eventcorresponds to the event information, the first image among the at leastone image that has been stored and associated with the event informationmay include: determining a plurality of images, among the at least oneimage, that includes the first image and that have in common at leastone of an object detected in the plurality of images or locationinformation indicating a location where the plurality of images wasobtained; and displaying the plurality of images including the firstimage.

In some implementations, displaying, on a display of the mobile terminaland based on the determination that the detected event corresponds tothe event information, the first image among the at least one image thathas been stored and associated with the event information may include:determining that the event detected on the mobile terminal is a firstevent; determining event occurrence information for each image of the atleast one image that has been stored and associated with the eventinformation; determining that the event occurrence information for thefirst image among the at least one image corresponds to the first eventdetected on the mobile terminal; and displaying the first image forwhich the event occurrence information corresponds to the first eventdetected on the mobile terminal.

In some implementations, displaying, on a display of the mobile terminaland based on the determination that the detected event corresponds tothe event information, the first image among the at least one image thathas been stored and associated with the event information may include:displaying a first portion of the first image at an edge of the displayof the mobile terminal; determining that a user touch-input has beenapplied to the displayed first portion of the first image; and based onthe determination that the user touch-input has been applied to thedisplayed first portion of the first image, displaying a second portionof the first image that includes both the first portion and a thirdportion of the first image different than the first portion.

In some implementations, the method may further include: based ondisplaying the second portion of the first image that includes both thefirst portion and the third portion of the first image different thanthe first portion, displaying a first user interface configured toreceive a user input to delete the first image, and a second userinterface configured to receive a user input to store the first image.

In some implementations, a mobile terminal includes a storage unit; adisplay unit; and a control unit communicative with the storage unit andthe display unit. The control unit may be configured to: obtain at leastone image; determine event information that is to be associated with theobtained at least one image; store, in the storage unit, the obtained atleast one image and information that associates the obtained at leastone image with the event information. The control unit may also beconfigured to detect an event on the mobile terminal; determine that thedetected event corresponds to the event information; and display, on thedisplay unit and based on the determination that the detected eventcorresponds to the event information, a first image among the at leastone image that has been stored and associated with the eventinformation.

In some implementations, the mobile terminal may further include: animage-capturing unit configured to capture the at least one image; andan input unit configured to receive a user input. The control unit mayfurther be configured to control the image-capturing unit to capture theat least one image through a first image-capturing mode of the mobileterminal. The control unit may be configured to determine the eventinformation that is to be associated with the at least one image by:receiving, through the input unit, the user input; and determining,based on the user input, the event information as being at least one oftime information indicating a time at which the at least one image is tobe displayed, location information indicating a location at which the atleast one image is to be displayed, and application informationindicating an application that is to be displayed together with the atleast one image.

In some implementations, the control unit may be configured to controlthe image-capturing unit to capture the at least one image through thefirst image-capturing mode of the mobile terminal by: controlling theimage-capturing unit to capture an image while a screen of the mobileterminal is turned off.

In some implementations, the mobile terminal may further include aninput unit configured to receive a user input. The control unit may beconfigured to control the display unit to display a plurality of imagesincluding the first image. The control unit may also be configured todetermine the event information that is to be associated with the atleast one image by: receiving, through the input unit, a user inputindicating the event information that is to be associated with the firstimage of the plurality of images; and determining, based on the userinput, the event information as including at least one of timeinformation indicating a time at which the first image is to bedisplayed, location information indicating a location at which the firstimage is to be displayed, and application information indicating anapplication to be displayed together with the first image.

In some implementations, the mobile terminal may further include animage-capturing unit configured to capture an image. The control unitmay further be configured to control the image-capturing unit to capturean image through a first image-capturing mode. The control unit may beconfigured to determine the event information that is to be associatedwith the at least one image by: determining the event information fromthe image that was captured through the first image-capturing mode.

In some implementations, determining the event information from theimage that was captured through the first image-capturing mode mayinclude: determining the event information based on at least one of anobject detected in the image, a text detected in the image, locationinformation indicating a location where the image was captured, timeinformation indicating a time at which the image was captured, orapplication information indicating an application that was executed whenthe image was captured.

In some implementations, the control unit may be configured to display,on the display unit and based on the determination that the detectedevent corresponds to the event information, the first image among the atleast one image that has been stored and associated with the eventinformation by: determining a plurality of images, among the at leastone image, that includes the first image and that have in common atleast one of an object detected in the plurality of images or locationinformation indicating a location where the plurality of images wasobtained; and controlling the display unit to display the plurality ofimages including the first image.

In some implementations, the control unit may be configured to display,on the display unit and based on the determination that the detectedevent corresponds to the event information, the first image among the atleast one image that has been stored and associated with the eventinformation by: determining that the event detected on the mobileterminal is a first event; determining event occurrence information foreach image of the at least one image that has been stored and associatedwith the event information; determining that the event occurrenceinformation for the first image among the at least one image correspondsto the first event detected on the mobile terminal; and controlling thedisplay unit to display the first image for which the event occurrenceinformation corresponds to the first event detected on the mobileterminal.

In some implementations, the mobile terminal may further include aninput unit configured to receive a user input. The control unit may beconfigured to display, on the display unit and based on thedetermination that the detected event corresponds to the eventinformation, the first image among the at least one image that has beenstored and associated with the event information by: controlling thedisplay unit to display a first portion of the first image at an edge ofa screen of the display unit; determining that a user input has beenreceived, through the input unit, to select the displayed first portionof the first image; and based on the determination that the user inputhas been received to select the displayed first portion of the firstimage, controlling the display unit to display a second portion of thefirst image that includes both the first portion and a third portion ofthe first image different than the first portion.

In some implementations, the control unit may further be configured to:based on controlling the display unit to display the second portion ofthe first image that includes both the first portion and the thirdportion of the first image different than the first portion, control thedisplay unit to display a first user interface configured to receive auser input to delete the first image, and a second user interfaceconfigured to receive a user input to store the first image.

All or part of the features described throughout this application can beimplemented as a computer program product including instructions thatare stored on one or more non-transitory machine-readable storage media,and that are executable on one or more processing devices. All or partof the features described throughout this application can be implementedas an apparatus, method, or electronic system that can include one ormore processing devices and memory to store executable instructions toimplement the stated functions.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features will beapparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of picture management;

FIG. 2 is a block diagram illustrating an example of a mobile terminalaccording to some implementations;

FIG. 3 is a flowchart illustrating an example of a method of operating amobile terminal according to some implementations;

FIG. 4 is a diagram illustrating an example of a mobile terminaloperating to obtain an image;

FIG. 5 is a diagram illustrating an example of capturing an imagethrough an image capturing technique according to some implementations;

FIGS. 6A, 6B, 6C, 7A, and 7B are diagrams illustrating examples ofassociating an image with a location where the image is to be displayed,according to a user input;

FIGS. 8A to 8E are diagrams illustrating examples of associating animage with a time at which the image is to be displayed, according to auser input;

FIGS. 9A to 9C are diagrams illustrating examples of associating animage with an application that is to be executed when the image isdisplayed, according to a user input;

FIGS. 10A and 10B are diagrams illustrating examples of associatingevent information to a plurality of images obtained by a mobileterminal, according to some implementations;

FIGS. 11A and 11B are diagrams illustrating examples of obtaining eventinformation that is to be associated with an image captured through aspecific image capturing mode, and associating the event information tothe image according to some implementations;

FIG. 12 is a diagram illustrating an example of a plurality of imagesobtained by a mobile terminal;

FIGS. 13A and 13B are diagrams illustrating examples of displaying aplurality of images by category, according to some implementations;

FIGS. 14A to 14C are diagrams illustrating examples of displaying aplurality of images based on an object detected in the images, accordingto some implementations;

FIG. 15 is a diagram illustrating an example of displaying a pluralityof images based on a location associated with the images and an objectdetected in the images, according to some implementations;

FIGS. 16A and 16B are diagrams illustrating examples of classifying anddisplaying images based on text that is detected in the images,according to some implementations;

FIGS. 17A to 17D are diagrams illustrating examples of displaying animage based on the event information indicating a location where theimage is displayed, according to some implementations;

FIGS. 18A and 18B are diagrams illustrating examples of displaying animage based on the event information indicating location information inthe form of a wireless signal information;

FIG. 19 is a diagram illustrating an example of displaying an imagebased on the event information indicating an application that is to bedisplayed with the image;

FIGS. 20A to 20C are diagrams illustrating examples of associating eventinformation and determining whether an event occurs on the basis ofpredetermined information stored in a mobile terminal and based on auser input, according to some implementations;

FIG. 21 is a diagram illustrating an example of displaying additionalinformation on an image displayed when an event occurs;

FIG. 22 is a diagram illustrating an example of displaying additionalinformation on a displayed image according to another implementation;

FIGS. 23A to 23C are diagrams illustrating examples of updating aschedule based on detecting that a text is included in an image capturedthrough a specific image-capturing mode; and

FIG. 24 is a diagram illustrating an example of performing textconversion from an image obtained through a specific image-capturingmode.

DETAILED DESCRIPTION

In some scenarios, a user may capture images of the user's environmentand use those images as part of an information management technique. Asan example, a user may capture an image of a parking zone identificationmark for a particular parking zone in a parking lot and, uponsubsequently returning to the same parking lot, the user may easily findthe parking zone by retrieving the stored image. As another example, auser may capture images of teaching material on a board or display, andthe user may subsequently retrieve those images for review of theteaching content. In general, users may capture images, either as asingle picture or multiple frames of a video, of an event of interestand subsequently retrieve those images for review of that event.

In such scenarios where images are used as an information managementtechnique, a user may desire particular functionality for interactingwith the images that differ from functionality for other images that arenot used for information management. For example, images that are usedfor information management may only be relevant to a user in aparticular scenario or timeframe, and may otherwise be irrelevant andappropriately deleted. Additionally, images used for informationmanagement may need to be easily and quickly accessed by a user inparticular scenarios or timeframes, and clearly distinguished from othertypes of images that are not used for information management.

As an example, consider a scenario where images are stored andclassified as an aggregate, as shown in FIG. 1. In this example, images51, 52, and 53 have been captured for information management but havebeen stored and classified together with other types of images that arenot used for information management. In this example, it may bedifficult for a user to easily access and retrieve images 51, 52, and 53to obtain the desired information.

In this example, a user may face additional delay in searching throughthe aggregate collection of images at a time or a place when the usermay need to quickly access the relevant information.

Additionally, images that are captured for information management maylose relevance after some period of time after a user accesses and usesthe information, such as images of a parking zone after the user hasleft a parking lot or images of teaching material after a user hascompleted a course. However, a user may face additional delay and effortin searching through an aggregate collection of images to find anddelete such irrelevant images.

Systems and techniques are described herein that enable a mobileterminal to classify, manage, and display images based on eventinformation that is associated with the images. The mobile terminal isconfigured to detect events that occur in a user's environment, anddisplay previously-stored images that have been associated with thedetect events.

In some implementations, the mobile terminal is configured to determineevent information that is to be associated with particular images. Whenthe mobile terminal detects that an event has occurred and determinesthat the event corresponds to a particular event information, the mobileterminal may display previously-stored images that have been associatedwith that event information.

Herein, a mobile terminal is described as one example of an electronicdevice that may be used to implement various systems and techniques.However, implementations are not limited to a mobile terminal and may beapplied to any suitable electronic device that is configured to captureimages.

FIG. 2 is a block diagram illustrating a mobile terminal according tosome implementations.

The mobile terminal 100 may include a wireless communication unit 110,an input unit 120, a capturing unit 130, a sensing unit 140, an outputunit 150, an interface unit 160, a storage unit 170, a control unit 180,and a power supply unit 190. In implementing a portable device,components shown in FIG. 1 are not necessary, so that a portable devicedescribed in this specification may include components less or more thanthe components listed above.

In more detail, the wireless communication unit 110 in the componentsmay include at least one module allowing wireless communication betweenthe mobile terminal 100 and a wireless communication system, between themobile terminal 100 and another mobile terminal 100, or between themobile terminal 100 and an external server. Additionally, the wirelesscommunication unit 110 may include at least one module connecting themobile terminal 100 to at least one network.

The wireless communication unit 110 may include at least one of abroadcast receiving module 111, a mobile communication module 112, awireless internet module 113, a short-range communication module 114,and a location information module 115.

The input unit 120 may include a camera 121 or an image input unit forimage signal input, a microphone 122 or an audio input unit for audiosignal input, and a user input unit 123 (for example, a touch key and amechanical key)) for receiving information from a user. Voice data orimage data collected by the input unit 120 are analyzed and processed asa user's control command.

The sensing unit 140 may include at least one sensor for sensing atleast one of information in a watch-type mobile terminal, environmentalinformation around a watch-type mobile terminal, and user information.For example, the sensing unit 140 may include at least one of aproximity sensor 141, an illumination sensor, a touch sensor 142, anacceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor,a motion sensor, an RGB sensor, an infrared (IR) sensor, a finger scansensor, an ultrasonic sensor, an optical sensor (for example, the camera121), a microphone (for example, the microphone 122), a battery gauge,an environmental sensor (for example, a barometer, a hygrometer, athermometer, a radiation sensor, a thermal sensor, and a gas sensor),and a chemical sensor (for example, an electronic noise, a healthcaresensor, and a biometric sensor). Moreover, the mobile terminal 100disclosed in this specification may combine information sensed by atleast two or more sensors among such sensors and may then utilize it.

The output unit 150 is used to generate a visual, auditory, or hapticoutput and may include at least one of a display unit 151, a soundoutput unit 152, a haptic module 153, and an optical output unit 154.The display unit 151 may be formed with a mutual layer structure with atouch sensor or formed integrally, so that a touch screen may beimplemented. Such a touch screen may serve as the user input unit 123providing an input interface between the mobile terminal 100 and a userand an output interface between the mobile terminal 100 and a user atthe same time.

The interface unit 160 may serve as a path to various kinds of externaldevices connected to the mobile terminal 100. The interface unit 160 mayinclude at least one of a wired/wireless headset port, an externalcharger port, a wired/wireless data port, a memory card port, a portconnecting a device equipped with an identification module, an audioInput/Output (I/O) port, a video I/O port, and an earphone port. Incorrespondence to that an external device is connected to the interfaceunit 160, the mobile terminal 100 may perform an appropriate controlrelating to the connected external device.

Additionally, the storage unit 170 may store data supporting variousfunctions of the mobile terminal 100. The storage unit 170 may store aplurality of application programs (for example, application programs orapplications) running on the mobile terminal 100 and also data andcommands for operations of the mobile terminal 100. At least part ofsuch an application program may be downloaded from an external serverthrough a wireless communication. Additionally, at least part of such anapplication program may be included in the mobile terminal 100 from thetime of shipment in order to perform a basic function (for example, anincoming call, a transmission function, and a message reception) of themobile terminal 100. Moreover, an application program may be stored inthe storage unit 170 and installed on the mobile terminal 100, so thatit may run to perform an operation (or a function) of the mobileterminal 100 by the control unit 180.

The control unit 180 may control overall operations of the mobileterminal 100 generally besides an operation relating to the applicationprogram. The control unit 180 may provide appropriate information orfunctions to a user or process them by processing signals, data, andinformation inputted/outputted through the above components or executingapplication programs stored in the storage unit 170.

Additionally, in order to execute an application program stored in thestorage unit 170, the control unit 180 may control at least part of thecomponents shown in FIG. 1. Furthermore, in order to execute theapplication program, the control unit 180 may combine at least two ofthe components in the mobile terminal 100 and may then operate it.

The power supply unit 190 may receive external power or internal powerunder a control of the control unit 180 and may then supply power toeach component in the mobile terminal 100. The power supply unit 190includes a battery and the battery may be a built-in battery or areplaceable battery.

At least part of the each component may operate cooperatively in orderto implement operations, controls, or control methods of a mobileterminal 100 according to various implementations described below.Additionally, the operations, controls, or control methods of a mobileterminal 100 may be implemented on the mobile terminal 100 by executingat least one application program stored in the storage unit 170.

Hereinafter, prior to examining various implementations implementedthrough the mobile terminal 100, the above-listed components aredescribed in more detail with reference to FIG. 1.

First, in describing the wireless communication unit 110, the broadcastreceiving module 111 of the wireless communication unit 110 may receivea broadcast signal and/or broadcast related information from an externalbroadcast management server through a broadcast channel. The broadcastchannel may include a satellite channel and a terrestrial channel. Atleast two broadcast receiving modules for simultaneous broadcastreception for at least two broadcast channels or broadcast channelswitching may be provided to the mobile terminal 100.

The mobile communication module 112 may transmit/receive a wirelesssignal to/from at least one of a base station, an external terminal, anda server on a mobile communication network established according to thetechnical standards or communication methods for mobile communication(for example, Global System for Mobile communication (GSM), CodeDivision Multi Access (CDMA), Code Division Multi Access 2000(CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only(EV-DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access(HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution(LTE), and Long Term Evolution-Advanced (LTE-A)).

The wireless signal may include various types of data according to avoice call signal, a video call signal, or text/multimedia messagetransmission.

The wireless internet module 113 refers to a module for wirelessinternet access and may be built in or external to the mobile terminal100. The wireless internet module 113 may be configured totransmit/receive a wireless signal in a communication network accordingto wireless internet technologies.

The wireless internet technology may include Wireless LAN (WLAN),Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance(DLNA), Wireless Broadband (WiBro), World Interoperability for MicrowaveAccess (WiMAX), High Speed Downlink Packet Access (HSDPA), High SpeedUplink Packet Access (HSUPA), Long Term Evolution (LTE), and Long TermEvolution-Advanced (LTE-A) and the wireless internet module 113transmits/receives data according at least one wireless internettechnology including internet technology not listed above.

From the viewpoint that wireless internet access by WiBro, HSDPA, HSUPA,GSM, CDMA, WCDMA, LTE, and LTE-A is achieved through a mobilecommunication network, the wireless internet module 113 performingwireless internet access through the mobile communication network may beunderstood as one type of the mobile communication module 112.

The short-range communication module 114 may support short-rangecommunication by using at least one of Bluetooth™, Radio FrequencyIdentification (RFID), Infrared Data Association (IrDA), Ultra-Wideband(UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity(Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB)technologies. The short-range communication module 114 may supportwireless communication between the mobile terminal 100 and a wirelesscommunication system, between the mobile terminal 100 and another mobileterminal 100, or between networks including the mobile terminal 100 andanother mobile terminal 100 (or an external server) through wirelessarea networks. The wireless area networks may be wireless personal areanetworks.

The short-range communication module 114 may detect (or recognize)another mobile device around the mobile terminal 100, which is capableof communicating with the mobile terminal 100. Furthermore, if the othermobile terminal is a device authenticated to communicate with the mobileterminal 100, the control unit 180 may transmit at least part of dataprocessed in the mobile terminal 100 to the other mobile terminalthrough the short-range communication module 114. Accordingly, a user ofthe other mobile terminal may use the data processed in the mobileterminal 100 through another mobile terminal. For example, accordingthereto, when a call is received by the mobile terminal 100, a user maytake a phone call through another mobile terminal or when a message isreceived by the mobile terminal 100, a user may check the receivedmessage through another mobile terminal.

The location information module 115 is a module for obtaining thelocation (or the current location) of a mobile terminal and itsrepresentative examples include a global positioning system (GPS) moduleor a Wi-Fi module. For example, the mobile terminal may obtain itsposition by using a signal transmitted from a GPS satellite through theGPS module. As another example, the mobile terminal may obtain itsposition on the basis of information of a wireless access point (AP)transmitting/receiving a wireless signal to/from the Wi-Fi module,through the Wi-Fi module. In some implementations, the positioninformation module 115 may perform a function of another module in thewireless communication unit 110 in order to obtain data on the locationof the mobile terminal alternatively or additionally. The locationinformation module 115 is a module for obtaining the position (or thecurrent position) of the mobile terminal and is not limited to a moduledirectly calculating and obtaining the position of the mobile terminal.

The microphone 122 processes external sound signals as electrical voicedata. The processed voice data may be utilized variously according to afunction (or an application program being executed) being performed inthe mobile terminal 100. Moreover, various noise canceling algorithmsfor removing noise occurring during the reception of external soundsignals may be implemented in the microphone 122.

The user input unit 123 is to receive information from a user and wheninformation is inputted through the user input unit 123, the controlunit may control an operation of the mobile terminal 100 to correspondto the inputted information. The user input unit 123 may include amechanical input means (or a mechanical key, for example, a button, adome switch, a jog wheel, and a jog switch at the front/back or side ofthe mobile terminal 100) and a touch type input means. As one example, atouch type input means may include a virtual key, a soft key, or avisual key, which is displayed on a touch screen through softwareprocessing or may include a touch key disposed at a portion other thanthe touch screen. Moreover, the virtual key or visual key may havevarious forms and may be disposed on a touch screen and for example, mayinclude graphic, text, icon, video, or a combination thereof.

Then, the mobile terminal 100, as a unit for inputting image information(or signal), audio information (or signal), data, or informationinputted from a user, may include a capturing unit 130. Moreover, thecapturing unit 130 may include one or more cameras. The capturing unit130 processes image frames such as a still image or a video obtained byan image sensor in a video call mode or a capturing mode. The processedimage frame may be displayed on the display unit 151 or stored in thestorage unit 170. Moreover, a plurality of cameras equipped in thecapturing unit 130 may be arranged in a matrix structure and through thecameras forming such a matrix structure, a plurality of imageinformation having various angles or focuses may be inputted to themobile terminal 100. Additionally, a plurality of cameras may bearranged in a stereo structure to obtain the left and right images forimplementing a three-dimensional image.

The sensing unit 140 may sense at least one of information in a mobileterminal, environmental information around a mobile terminal, and userinformation and may then generate a sensing signal correspondingthereto. On the basis of such a sensing signal, the control unit 180 maycontrol the drive or control of the mobile terminal 100 or may performdata processing, functions, or operations relating to an applicationprogram installed in the mobile terminal 100. Representative sensorsamong various sensors included in the sensing unit 140 will be describedin more detail.

First, the proximity sensor 141 refers to a sensor detecting whetherthere is an object approaching a predetermined detection surface orwhether there is an object around by using the strength of anelectromagnetic field or infrared, without mechanical contact. Theproximity sensor 141 may disposed in an inner area of a mobile terminalsurrounded by the touch screen or around the touch screen.

Examples of the proximity sensor 141 may include a transmission-typephotoelectric sensor, a direct reflective-type photoelectric sensor, amirror reflective-type photoelectric sensor, a high-frequencyoscillation-type proximity sensor, a capacitive-type proximity sensors,a magnetic-type proximity sensor, and an infrared proximity sensor. Ifthe touch screen is a capacitive type, the proximity sensor 141 may beconfigured to detect the proximity of an object by changes in anelectric field according to the proximity of the object havingconductivity. In this case, the touch screen (or a touch sensor) itselfmay be classified as a proximity sensor.

Moreover, for convenience of description, an action for recognizing theposition of an object on the touch screen as the object is close to thetouch screen without contacting the touch screen is called “proximitytouch” and an action that the object actually contacts the touch screenis called “contact touch”. A position that an object isproximity-touched on the touch screen is a position that the objectvertically corresponds to the touch screen when the object isproximity-touched. The proximity sensor 141 may detect a proximity touchand a proximity touch pattern (for example, a proximity touch distance,a proximity touch direction, a proximity touch speed, a proximity touchtime, a proximity touch position, and a proximity touch movement state).Moreover, the control unit 180 processes data (for information)corresponding to a proximity touch operation and a proximity touchpattern, detected through the proximity sensor 141, and furthermore, mayoutput visual information corresponding to the processed data on thetouch screen. Furthermore, according to whether a touch for the samepoint on the touch screen is a proximity touch or a contact touch, thecontrol unit 180 may control the mobile terminal 100 to processdifferent operations or data (or information).

The touch sensor 142 detects a touch (or a touch input) applied to thetouch screen (or the display unit 151) by using at least one of varioustouch methods, for example, a resistive film method, a capacitivemethod, an infrared method, an ultrasonic method, and a magnetic fieldmethod.

For example, the touch sensor 142 may be configured to convert apressure applied to a specific portion of the touch screen or changes incapacitance occurring at a specific portion into electrical inputsignals. The touch sensor 142 may be configured to detect a position andarea that a touch target applying a touch on the touch screen touchesthe touch sensor 142, a pressured when touched, and a capacitance whentouched. Here, the touch target, as an object applying a touch on thetouch sensor, may be a finger, a touch pen, a stylus pen, or a pointer,for example.

In such a way, when there is a touch input on the touch sensor 142,signal(s) corresponding thereto is/are sent to a touch controller. Thetouch controller processes the signal(s) and then transmitscorresponding data to the control unit 180. Therefore, the control unit180 may recognize which area of the display unit 151 is touched. Herein,the touch controller may be an additional component separated from thecontrol unit 180 or may be the control unit 180 itself.

Moreover, the control unit 180 may perform different controls or thesame control according to types of a touch target touching the touchscreen (or a touch key equipped separated from the touch screen).Whether to perform different controls or the same control according totypes of a touch target may be determined according to a currentoperation state of the watch-type mobile terminal 100 or an applicationprogram in execution.

Moreover, the above-mentioned touch sensor 142 and proximity sensor 141are provided separately or combined and may thus sense various types oftouches, for example, short (or tap) touch), long touch, multi touch,drag touch, flick touch, pinch-in touch, pinch-out touch, swipe touch,and hovering touch on the touch screen.

The sensing unit 140 may include a movement sensing unit. The movementsensing unit, for example, a gyro sensor, may sense a movement of amobile terminal, for example, a movement direction, a movement angle, amovement speed, a current location, a rotation direction, and a rotationangle of a mobile terminal, and output a signal corresponding to themovement of the mobile terminal to the control unit 180. The controlunit 180 may detect movement related information from signals generatedfrom the gyro sensor, for example, a movement direction, a movementangle, a movement speed, an intensity, a current location, a rotationdirection, and a rotation angle of a mobile terminal.

Moreover, the capturing unit 130 may include at least one of a camerasensor (for example, CCD and CMOS), a photo sensor (or an image sensor),and a laser sensor.

The capturing unit 130 and the laser sensor may be combined to detect atouch of a detection target for a three-dimensional image. The photosensor may be stacked on a display device and is configured to scan amovement of a detection target close to the touch screen. In moredetail, the photo sensor mounts a photo diode and a transistor (TR) in arow/column and scans content disposed on the photo sensor by using anelectrical signal changing according to an amount of light applied tothe photo diode. That is, the photo sensor may calculate the coordinatesof a detection target according to the amount of change in light andthrough this, may obtain the position information of the detectiontarget.

The display unit 151 may display (output) information processed in themobile terminal 100. For example, the display unit 151 may displayexecution screen information of an application program running on themobile terminal 100 or user interface (UI) and graphic user interface(GUI) information according to such execution screen information.

The sound output unit 152 may output audio data received from thewireless communication unit 110 or stored in the storage unit 170 in acall signal reception or call mode, a recording mode, a voicerecognition mode, or a broadcast reception mode. The sound output unit152 may output a sound signal relating to a function (for example, acall signal reception sound and a message reception sound) performed bythe mobile terminal 100. The sound output unit 152 may include areceiver, a speaker, and a buzzer.

The haptic module 153 generates various haptic effects that a user canfeel. A representative example of a haptic effect that the haptic module153 generates is vibration. The intensity and pattern of vibrationgenerated by the haptic module 153 may be controlled by a user'sselection or a setting of a control unit. For example, the haptic module153 may synthesize and output different vibrations or output differentvibrations sequentially.

The haptic module 153 may generate various haptic effects, for example,effects by a pin arrangement moving vertical to a contact skin surface,injection power or suction power of air through an injection port or asuction port, rubbing a skin surface, electrode contact, stimulus ofelectrostatic force and effects by the reproduction of cold/warm senseby using a device absorbing or emitting heat.

The haptic module 153 may be implemented to deliver a haptic effectthrough a direct contact and also allow a user to feel a haptic effectthrough a muscle sense such as a finger or an arm. The haptic module 153may be more than two according to a configuration aspect of the mobileterminal 100.

The optical output unit 154 outputs a signal for notifying eventoccurrence by using light of a light source of the mobile terminal 100.An example of an event occurring in the mobile terminal 100 includesmessage reception, call signal reception, missed calls, alarm, schedulenotification, e-mail reception, and information reception through anapplication.

A signal outputted from the optical output unit 154 is implemented as amobile terminal emits single color of multi-color to the front or theback. The signal output may be terminated when a mobile terminal detectsuser's event confirmation.

The interface unit 160 may serve as a path to all external devicesconnected to the mobile terminal 100. The interface unit 160 may receivedata from an external device, receive power and deliver it to eachcomponent in the mobile terminal 100, or transmit data in the mobileterminal 100 to an external device. For example, the interface unit 160may include at least one of a wired/wireless headset port, an externalcharger port, a wired/wireless data port, a memory card port, a portconnecting a device equipped with an identification module, an audio I/Oport, a video I/O port, and/or an earphone port.

Moreover, the identification module, as a chip storing variousinformation for authenticating usage authority of the mobile terminal100, may include a user identity module (UIM), a subscriber identitymodule (SIM), and a universal subscriber identity module (USIM). Adevice equipped with an identification module (hereinafter referred toas an identification device) may be manufactured in a smart card form.Accordingly, the identification device may be connected to the terminal100 through the interface unit 160.

Additionally, when the mobile terminal 100 is connected to an externalcradle, the interface unit 160 may become a path through which power ofthe cradle is supplied to the mobile terminal 100 or a path throughwhich various command signals inputted from the cradle are delivered tothe mobile terminal 100 by a user. The various command signals or thepower inputted from the cradle may operate as a signal for recognizingthat the mobile terminal 100 is accurately mounted on the cradle.

The storage unit 170 may store a program for an operation of the controlunit 180 and may temporarily store input/output data (for example, aphone book, a message, a still image, and a video). The memory 170 maystore data on various patterns of vibrations and sounds outputted duringa touch input on the touch screen.

The storage unit 170 may include at least one type of storage mediumamong flash memory type, hard disk type, Solid State Disk (SSD) type,Silicon Disk Drive (SDD) type, multimedia card micro type, card typememory (for example, SD or XD memory type), random access memory (RAM)type, static random access memory (SRAM) type, read-only memory (ROM)type, electrically erasable programmable read-only memory (EEPROM) type,programmable read-only memory (PROM) type, magnetic memory type,magnetic disk type, and optical disk type. The mobile terminal 100 mayoperate in relation to a web storage performing a storage function ofthe storage unit 170 on internet.

Moreover, as mentioned above, the control unit 180 may controloperations relating to an application program and overall operations ofthe mobile terminal 100 in general. For example, if a state of themobile terminal 100 satisfies set conditions, the control unit 180 mayexecute or release a lock state limiting an output of a control commandof a user for applications.

Additionally, the control unit 180 may perform a control or processingrelating to a voice call, data communication, and a video call mayperform pattern recognition processing for recognizing handwriting inputor drawing input on the touch screen as a text and an image,respectively. Furthermore, the control unit 180 may use at least one ora combination of the above components to perform a control in order toimplement various implementations described below on the watch-typemobile terminal 100.

The power supply unit 190 may receive external power or internal powerunder a control of the control unit 180 and may then supply power for anoperation of each component. The power supply unit 190 includes abattery. The battery is a rechargeable built-in battery and may bedetachably coupled to a terminal body in order for charging.

Additionally, the power supply unit 190 may include a connection portand the connection port may be configured as one example of theinterface unit 160 to which an external charger supplying power forcharging of the battery is electrically connected.

As another example, the power supply unit 190 may be configured tocharge a battery through a wireless method without using the connectionport. In this case, the power supply unit 190 may receive power from anexternal wireless power transmission device through at least one of aninductive coupling method based on a magnetic induction phenomenon, anda magnetic resonance coupling method based on an electromagneticresonance phenomenon.

Then, a communication system using the mobile terminal 100 is describedaccording to an implementation.

First, the communication system may use different wireless interfacesand/or physical layers. For example, a wireless interface available tothe communication system may include Frequency Division Multiple Access(FDMA), Time Division Multiple Access (TDMA), Code Division MultipleAccess (CDMA), and Universal Mobile Telecommunications Systems (UMTS)(especially, Long Term Evolution (LTE), Long Term Evolution-Advanced(LTE-A), and Global System for Mobile Communications (GSM)).

Hereinafter, for convenience of description, description is made limitedto CDMA. However, implementations are applicable to any suitablecommunication system, such as Orthogonal Frequency Division Multiplexing(OFDM) wireless communication systems in addition to CDMA wirelesscommunication systems.

The CDMA wireless communication system may include at least one terminal100, at least one base station (BS) (it may be referred to as Node B orEvolved Node B), at least one base station controllers (BSCs), and amobile switching center (MSC). MSC may be configured to be connected toPublic Switched Telephone Network (PSTN) and BSCs. BSCs may be connectedbeing paired with a BS through a backhaul line. The backhaul line may beprovided according to at least one of E1/T1, ATM, IP, PPP, Frame Relay,HDSL, ADSL, and xDSL. Accordingly, a plurality of BSCs may be includedin a CDMA wireless communication system.

Each of a plurality of BSs may include at least one sensor and eachsensor may include an omni-directional antenna or an antenna indicatinga specific radial direction from a BS. Additionally, each sensor mayinclude at least two antennas in various forms. Each BS may beconfigured to support a plurality of frequency allocations and each ofthe plurality of frequency allocations may have a specific spectrum (forexample, 1.25 MHz, 5 MHz, and so on).

The intersection of a sector and a frequency allocation may be referredto as a CDMA channel. A BS may be referred to as a Base StationTransceiver Subsystem (BTS). In such a case, one BSC and at least one BStogether may be referred to as “BS”. A BS may also represent “cellsite”. Additionally, each of a plurality of sectors for a specific BSmay be referred to as a plurality of cell sites.

A Broadcasting Transmitter (BT) transmits broadcast signals to theterminals 100 operating in a system. The broadcast reception module 111shown in FIG. 1 is provided in the terminal 100 for receiving broadcastsignals transmitted from the BT.

Additionally, GPS may be linked to a CDMA wireless communication systemin order to check the location of the mobile terminal 100. Then, asatellite helps obtaining the location of the mobile terminal 100.Useful location information may be obtained by at least one satellite.Herein, the location of the mobile terminal 100 may be traced by usingall techniques for tracing the location in addition to GPS trackingtechnique. Additionally, at least one GPS satellite may be responsiblefor satellite DMB transmission selectively or additionally.

The location information module 115 in a mobile terminal is fordetecting and calculating the position of the mobile terminal and itsrepresentative example may include a GPS module and a WiFi module. Insome implementations, the position information module 115 may perform afunction of another module in the wireless communication unit 110 inorder to obtain data on the location of the mobile terminalalternatively or additionally.

The GPS module 115 may calculate information on a distance from at leastthree satellites and accurate time information and then applytriangulation to the calculated information, in order to accuratelycalculate the 3D current location information according to latitude,longitude, and altitude. A method for calculating location and timeinformation by using three satellites and correcting errors of thecalculated location and time information by using another one satelliteis being widely used. Additionally, the GPS module 115 may speedinformation as continuously calculating the current location in realtime. However, it is difficult to accurately measure the location of amobile terminal by using a GPS module in a shadow area of a satellitesignal such as a room. Accordingly, in order to compensate for themeasurement of a GPS method, a WiFi Positioning System (WPS) may beutilized.

WPS is a technique for tracking the location of the mobile terminal 100by using a WiFi module in the mobile terminal 100 and a wireless AccessPoint (AP) for transmitting or receiving wireless signals to or from theWiFi module and may mean a Wireless Local Area Network (WLAN) basedlocation measurement technique using WiFi.

A WiFi location tracking system may include a WiFi location measurementserver, a mobile terminal 100, a wireless AP connected to the mobileterminal 100, and a database for storing arbitrary wireless APinformation.

The mobile terminal 100 in access to a wireless AP may transmit alocation information request message to a WiFi location measurementserver.

The WiFi location measurement server extracts information of a wirelessAP connected to the mobile terminal 100 on the basis of a locationinformation request message (or signal) of the mobile terminal 100.Information of a wireless AP connected to the mobile terminal 100 may betransmitted to the WiFi location measurement server through the mobileterminal 100 or may be transmitted from a wireless AP to a WiFi locationmeasurement server.

Based on the location information request message of the mobile terminal100, the extracted information of a wireless AP may be at least one ofMAC Address, Service Set Identification (SSID), Received Signal StrengthIndicator (RSSI), Reference Signal Received Power (RSRP), ReferenceSignal Received Quality (RSRQ), channel information, Privacy, NetworkType, Signal Strength, and Noise Strength.

As mentioned above, the WiFi position measurement server may extractwireless AP information corresponding to a wireless AP that the mobileterminal 100 access from a pre-established database by using informationof the wireless AP connected to the mobile terminal 100. At this point,information of arbitrary wireless APs stored in the database mayinformation such as MAC Address, SSID, channel information, Privacy,Network Type, latitude and longitude coordinates of a wireless AP, abuilding name where a wireless AP is located, the number of floors,indoor detailed location (GPS coordinates available), the address of theowner of an AP, and phone numbers. At this point, in order to remove amobile AP or a wireless AP provided using illegal MAC address during ameasurement process, a WiFi location measurement server may extract onlya predetermined number of wireless AP information in high RSSI order.

Then, the WiFi location measurement server may extract (or analyze) thelocation information of the mobile terminal 100 by using at least onewireless AP information extracted from the database. By comparing theincluded information and the received wireless AP information, locationinformation of the mobile terminal 100 is extracted (or analyzed).

As a method of extracting (or analyzing) the location information of themobile terminal 100, a Cell-ID method, a finger-print method, atriangulation method, and a landmark method may be used.

The Cell-ID method is a method for determining the location of awireless AP having the strongest signal intensity in neighbor wirelessAP information that a mobile terminal collects as the location of themobile terminal. Implementation is simple, no additional cost isrequired, and location information is obtained quickly but when theinstallation density of wireless APs is low, measurement precision ispoor.

The finger-print method is a method for collecting signal intensityinformation by selecting a reference location from a service area andestimating the location through signal intensity information transmittedfrom a mobile terminal on the basis of the collected information. Inorder to use the finger-print method, there is a need to provide adatabase for storing propagation characteristics in advance.

The triangulation method is a method for calculating the location of amobile terminal on the basis of a distance between coordinates of atleast three wireless APs and a mobile terminal. In order to measure adistance between a mobile terminal and a wireless AP, a signal intensityconverted into distance information, Time of Arrival (ToA), TimeDifference of Arrival (TDoA), and Angle of Arrival (AoA) may be used.

The landmark method is a method for measuring the location of a mobileterminal by using a landmark transmitter knowing the location.

In addition to the listed methods, a variety of algorithms may beutilized as methods for extracting (or analyzing) the locationinformation of a mobile terminal.

As the extracted location information of the mobile terminal 100 istransmitted to the mobile terminal 100 through the WiFi locationmeasurement server, the mobile terminal 100 may obtain the locationinformation.

As connected to at least one wireless AP, the mobile terminal 100 mayobtain location information. At this point, the number of wireless APs,which are required for obtaining the location information of the mobileterminal 100, may vary according to a wireless communication environmentwhere the mobile terminal 100 is located.

Moreover, some components of the mobile terminal 100 may be omitted orchanged.

Moreover, an input unit 120 may further include a sensing unit 140

Moreover, various implementations below may be implemented in a computeror device similar thereto readable medium by using software, hardware,or a combination thereof.

FIG. 3 is a flowchart illustrating a method of operating a mobileterminal according to some implementations.

Referring to FIG. 3, a method of operating the mobile terminal 100includes obtaining an image in operation S310, and matching the obtainedimage to event information and storing the image matched to the eventinformation in operation S330. In some implementations, storing theimage matched to the event information may include storing the image,the event information, and information that matches the image with theevent information. The method further includes detecting that a specificevent occurs, and displaying an image corresponding to the specificevent among at least one image matched to each of at least one eventinformation and stored in operation S350, and displaying a menu forreceiving a user input for deleting a displayed image and a menu forreceiving a user input for storing a displayed image in operation S370.

Among operations, operation S310 for obtaining an image is describedwith reference to FIG. 4.

FIG. 4 is a view illustrating a method of the mobile terminal 100 toobtain an image.

Referring to FIG. 4, the mobile terminal 100 may receive an image fromthe outside through a mobile communication network and a wirelessinternet network. Additionally, the mobile terminal 100 may obtain animage by receiving the image from an external device through thewireless communication unit 110 or the interface unit 160.

Moreover, the mobile terminal 100 may obtain an image by capturing theimage through the capturing unit 130. In more detail, when a user inputfor capturing an image is received, the control unit 180 may control thecapturing unit 130 to capture an image received through the capturingunit 130.

Moreover, the control unit 180 may classify at least part of theobtained images as an image for providing information and manage it. Forexample, the control unit 180 may control the display unit 151 todisplay only an image providing information among the obtained imagesaccording to a specific user input and delete an image for providinginformation, which is stored in the storage unit 170, automaticallyafter a specific time point elapses after the acquisition. Herein, animage for providing information refers to all images used for a user toobtain information and may be distinguished from a just viewing picture.

Additionally, an obtained image may be stored as an image for providinginformation as a user a capture an image through a specific imagecapturing mode. Additionally, when a general capturing technique insteadof a specific capturing technique is performed, a user may match anobtained image to event information and store it as an image forproviding information. Additionally, when an image obtained as capturingis performed through a general image capturing mode, since eventinformation is matched according to place information of the obtainedimage, the obtained image may be stored as an image for providinginformation.

Referring to FIG. 3 again, the method of operating a mobile terminal mayinclude matching the obtained image and the event information andstoring it in operation S330. This will be described with reference toFIGS. 5 and 11.

FIGS. 5 to 9 are views illustrating a method of matching an image, whichis captured through a specific image capturing mode and stored, andevent information by a user input.

FIG. 5 is a view illustrating capturing an image through a specificimage capturing mode according to some implementations.

The control unit 180 may obtain an image captured through a specificimage capturing mode in the mobile terminal 100.

Herein, the specific image capturing mode may be a capturing modethrough “quick shoot”. Herein, “quick shoot” refers to a capturing modefor executing a capture application and capturing an image by clicking aspecified button 510 at the rear of the mobile terminal 100 two timeswhile the screen of the mobile terminal 100 is not turned on. Moreover,the capturing unit 130 may include a first camera mounted at the frontof the mobile terminal 100, that is, the same surface as a touch screenand a second camera mounted at the rear of the mobile terminal 100, thatis, an opposite surface to the touch screen, and when an image iscaptured through quick shoot, capturing is performed by using the firstcamera.

Moreover, the specific capturing mode is not limited to “quick shoot”.For example, the control unit 180 displays a capture menu for providinginformation on a screen during image capturing and captures an imageaccording to a user input for selecting the capture menu for providinginformation so that it obtains an image captured through a specificcapturing mode.

For another example, by capturing an image as a user input for touchingthe image capture menu for more than a predetermined time is received,the control unit 180 may obtain an image captured through a specificcapturing mode.

For another example, when the capturing unit 130 includes a first cameraand a second camera, the control unit 180 may obtain an image capturedthrough the first camera as an image captured through a specificcapturing mode. Herein, the first camera may be a camera mounted at thefront of the mobile terminal 100 and the second camera may be a cameramounted at the rear of the mobile terminal 100. Additionally, the firstcamera may be a camera mounted at the rear of the mobile terminal 100and the second camera may be a camera mounted at the front of the mobileterminal 100.

For another example, the specific capturing mode may be a mode ofcapturing an image while the screen of the mobile terminal 100 is turnedoff. In more detail, when the screen of the mobile terminal 100 isturned off, as a user input for a specific button of the mobile terminal100 is received, the control unit 180 may control the capturing unit 130to capture an image.

For another example, the specific capturing mode may be a mode ofcapturing an image while a camera application is not in execution. Inmore detail, while a camera application is not in execution, as aspecific user input is received by the mobile terminal 100, the controlunit 180 may control the capturing unit 130 to capture an image.

In such a way, the system enables classification of an image that iscaptured through a specific capturing mode so that a user may easilyaccess images and associated information.

Hereinafter, for convenience of description, the specific capturing modeis a capturing mode by “quick shoot”. However, implementations are notlimited thereto and any suitable capturing mode distinguished from thecapturing mode described with reference to FIG. 5 and a generalcapturing mode for capturing an image for recollection and viewing maybe performed as a specific capturing mode.

In some implementations, the matching of the obtained image to the eventinformation and the storing of it in operation S330 may includereceiving a user input for inputting the event information, matching theimage captured through a specific capturing mode to the inputted eventinformation and storing the image matched to the inputted eventinformation. In such a scenario, the event information inputted by auser may be at least one of information on a time point that an obtainedimage is displayed, information on a place where an obtained image isdisplayed, and information on an application to be displayed togetherwith an image obtained when the application is executed.

FIGS. 6A, 6B, 6C, 7A, and 7B are diagrams illustrating an example ofmatching information on a location where an image is displayed accordingto a user input.

As shown in FIG. 6A, when image-capturing is performed through aspecific capturing mode, the control unit 180 may obtain an imagecaptured through the specific capturing mode and, as shown in FIG. 6B,control the display unit 151 to display the obtained image 610.

Moreover, the control unit 180 may store the obtained image 610 in thestorage unit 170.

In some implementations, as a user input for dragging the displayedimage 610 is received, the control unit 180, as shown in FIG. 6C, maydisplay a UI object 620 for inputting event information through a uservoice input. Additionally, when a user voice input for inputting eventinformation is received through the input unit 120, the control unit 180may match the received event information to the obtained image 610 andstore it in the storage unit 170. For example, the control unit 180 maystore the image 610 matched to the event information by matching theevent information to the image 610 and storing the image 1110 and theevent information or by matching the event information that was receivedin the user voice input to a pre-stored image 610 and event information.

Herein, the event information inputted by the user may be information ona place where the obtained image is displayed. In more detail, when auser input for specifying a place where the displayed image 610 isdisplayed is received, the control unit 180 may match locationinformation on a specific place specified by a user as event informationto the displayed image 610 and store it.

For example, as shown in FIG. 6C, if a user input for displaying thedisplayed image 610 at a place where the image is captured is received,the control unit 180 may obtain location information on a location wherethe image is captured and match the obtained location information asevent information to the displayed image 610 and store it.

In more detail, the control unit 180 may obtain location information ofthe mobile terminal 100 at a time point that an image is captured on thebasis of GPS information of the mobile terminal 100 at a time point thatthe image is captured and information on a wireless signal detected bythe mobile terminal 100. For example, when a place where an image iscaptured is a parking lot, the control unit 180 may obtain locationinformation of the mobile terminal 100 when an image is captured on thebasis of GPS coordinates of the parking lot or wireless signalinformation such as Bluetooth signals transmitted from an electronicdevice in a vehicle parked in the parking lot, BLE signals transmittedfrom Beacon, WiFi signals transmitted from a wireless Access Point (AP),and signals transmitted during an NFC process.

For another example, if a user input for displaying the displayed image610 on another place instead of a place where the image is captured isreceived, the control unit 180 may match location information on alocation specified by a user as event information to the displayed image610 and store it. For example, as shown in FIG. 6C, when a user inputfor displaying an image captured at a parking lot on a school isreceived, the control unit 180 may match location information on theschool as event information to the displayed image 610 and store it.

Herein, location information on a specific place specified by a user maybe location information obtained by using context information. In moredetail, the control unit 180 may learn location information on aspecific place on the basis of location information of the mobileterminal 100 and context information obtained from information inputtedby a user.

For example, the control unit 180 may learn that a school is located ata specific location on the basis of a user input related to “school” andlocation information of the mobile terminal 100 at the school. Then, ifa user input for displaying the displayed image 610 again at the“school” is received, the control unit 180 may match locationinformation on the “school” as event information to the displayed image610 and store it.

Moreover, the event information inputted by the user may be informationon a place where an obtained image is displayed and herein, theinformation on the place is not limited to one place located in aspecific region.

An example of this will be described in detail with reference to FIGS.7A and 7B. In this implementation, as shown in FIG. 7A, eventinformation is matched to allow a discount coupon image 710 used at aplurality of “convenience stores A” to be used at the plurality of“convenience stores A.”

A “place” specified by a user may refer to a single place located atspecific coordinates and also a plurality of places corresponding toeach other. For example, a “convenience store A” that is a placespecified by a user as shown in FIG. 7A may include a plurality ofplaces corresponding to each other, that is, all chain stores of theconvenience store A.

Moreover, as shown in FIG. 7B, after a touch on a specific menu 720 isdetected, when a user input for specifying a place where the displayedimage 710 is displayed is received, the control unit 180 may matchlocation information on a place specified by a user as event informationto the displayed image 710 and store it.

In this case, the location information may be wireless signalinformation. In more detail, the control unit 180 may match wirelesssignal information commonly transmitted from a plurality of placescorresponding to each other as location information on a place specifiedby a user to the displayed image 710 and store it.

For example, the control unit 180 may match wireless signals (forexample, unique BLE signals from the “convenience store A”) commonlytransmitted from the plurality of “convenience stores A” located in aplurality of regions as location information on a place specified by auser to the displayed image 710 and store it.

In this case, location information on a place specified by a user may belocation information obtained by using context information. For example,the control unit 180 may learn wireless signals transmitted from the“convenience store A” on the basis of a “convenience store A” relateduser input, a user's visit history on the “convenience store A”, andwireless signal information sensed by the mobile terminal 100 whenvisiting the “convenience store A”. Then, when a user input fordisplaying the displayed image 710 again at the “convenience store A” isreceived, the control unit 180 may match wireless signals transmittedfrom the “convenience store A” as event information to the displayedimage 710 and store it.

FIGS. 8A to 8E are diagrams illustrating examples of matchinginformation on a time point that an image is displayed according to auser input to an image.

As shown in FIG. 8A, when capturing is performed through a specificcapturing mode, the control unit 180 may obtain an image capturedthrough the specific capturing mode, and as shown in FIG. 6B, controlthe display unit 151 to display the obtained image 810.

Moreover, as a user input for dragging the displayed image 810 isreceived, the control unit 180, as shown in FIG. 8C, may display a UIobject 820 for inputting event information through a user voice input.Additionally, when a user voice input for inputting event information isreceived through the input unit 120, the control unit 180 may match thereceived event information to the obtained image 810 and store it in thestorage unit 170.

Herein, the event information inputted by the user may be information ona time point that the obtained image is displayed. In more detail, whena user input for specifying a time point that the displayed image 810 isdisplayed is received, the control unit 180 may match information on atime point specified by a user as event information to the displayedimage 810 and store it.

For another example, when a user input for displaying the displayedimage 810 on “November 26th” is received, the control unit 180 may matchtime point information of “November 26th” as event information to thedisplayed image 810 and store it.

Moreover, the control unit 180 may obtain information on a time pointthat the image 810 is displayed by using a user input and informationpre-stored in the mobile terminal 100 and match the obtained time pointinformation as event information to the displayed image 810 and storethe obtained information on a time point. For example, when a user inputfor “show this picture again on my birthday” is received, the controlunit 180 may match time point information of “November 26th” as eventinformation to the displayed image 810 and store the time pointinformation of “November 26th” by using information that the user'sbirthday is “November 26th” pre-stored in the storage unit 170.

Moreover, when the information on the time point that the image 810 isdisplayed again is not obtained through a user input, for example, asshown in FIG. 8C, when the display time point of the image 810 is notspecified after a user input for “show this picture again later” isreceived, as shown in FIG. 8D, the control unit 180 may output a queryfor obtaining a time point that the image 810 is displayed again.

Furthermore, this implementation describes that a query for obtaining atime point that the image 810 is displayed again but is not limitedthereto and a query outputted from the mobile terminal 100 may be aquery for obtaining a place where an image is displayed again or a queryfor obtaining information on an application to be displayed togetherwith an image.

Moreover, as shown in FIG. 8E, when a user input for inputting eventinformation is received in response to a query outputted from the mobileterminal 100, the control unit 180 may obtain event information from thereceived user input and match the obtained event information to thedisplayed image 810 and store it.

FIGS. 9A to 9C are diagrams illustrating examples of matchingapplication information to be displayed together with an image duringapplication execution to the image according to a user input.

As shown in FIG. 9A, when capturing is performed through a specificcapturing mode, the control unit 180 may obtain an image capturedthrough the specific capturing mode, and as shown in FIG. 9B, controlthe display unit 151 to display the obtained image 910.

Moreover, as a user input for dragging the displayed image 910 isreceived, the control unit 180, as shown in FIG. 9C, may display a UIobject 920 for inputting event information through voice. Additionally,when a user voice input for inputting event information is receivedthrough the input unit 120, the control unit 180 may match the receivedevent information to the obtained image 910 and store it in the storageunit 170.

Herein, the event information inputted by the user may be information ona specific application to be displayed together with the obtained image910 during application execution. In more detail, when a user input forspecifying a specific application to be displayed together with thedisplayed image 910 during application execution, the control unit 180may match information on an application specified by a user as eventinformation to the displayed image 910 and store it.

For example, when a user input for displaying the obtained image 910during a “bank B application” execution is received, the control unit180 may match “bank B application” information as event information tothe displayed image 910 and store it.

Moreover, although it is described with reference to FIGS. 6A to 9C thatevent information is matched by receiving a user voice input andvoice-recognizing the received user voice input, implementations are notlimited thereto and any suitable user input method for specifying a timepoint, a place, and/or an application that an obtained image isdisplayed may be used.

For example, when capturing is performed through a specific mode asshown in FIG. 5, the control unit 180 may obtain an image for specifyinga place where an obtained image is displayed again. A displayed imagemay include location information on a place where an image is obtainedor a place frequently visited by a user and when a user input forselecting a specific place among displayed places is received, thecontrol unit 180 may match location information on a place selected by auser to an obtained image and store it in the storage unit 170.

For another example, when capturing is performed through a specific modeas shown in FIG. 5, the control unit 180 may display an image forspecifying a time point that an obtained image is displayed again andmatch time point information inputted by a user to the obtained imageand store it in the storage unit 170.

For another example, when capturing is performed through a specific modeas shown in FIG. 5, the control unit 180 may display a list ofapplications installed on the mobile terminal 100 and match a specificapplication selected by a user to the obtained image and store it in thestorage unit 170.

Additionally, although it is described with reference to FIGS. 6A to 9Cthat when an image captured through a specific capturing mode isobtained, the obtained image is displayed and as a user input fordragging the displayed image is received, a user input for inputtingevent information is received, implementations are not limited thereto.

For example, when an image is obtained through a specific capturingmode, the control unit 180 may receive a user input for inputting eventinformation may be received, without a process for displaying theobtained image and a process for receiving a user's drag input.

For another example, when an image captured through a specific capturingmode is obtained and a specific menu displayed on a screen is selectedin a state of displaying the obtained display, a user input forinputting event information may be received.

In such a way, according to some implementations, by inputting eventinformation to an image captured through a specific capturing mode, animage for providing information may be easily displayed when a userwants to obtain information.

FIGS. 10A and 10B are diagrams illustrating examples of matching eventinformation to a plurality of images obtained by a mobile terminal orsetting whether to provide them according to some implementations.

Referring to FIG. 10A, the control unit 180 may obtain a plurality ofimages and store the plurality of images. The control unit 180 maycontrol the display unit 151 to display a plurality of images obtainedby the mobile terminal 100. Herein, the plurality of images may includeat least one of an image received from the outside through a mobilecommunication network or a wireless internet network, an image receivedfrom an external device through the wireless communication unit 110 orthe interface unit 160, an image obtained through the specific capturingmode of FIG. 5, and an image obtained through another capturing modeinstead of a specific capturing mode.

Moreover, icons 1011, 1021, and 1031 for identifying whether eventinformation is matched may be displayed together on an image where eventinformation is matched and stored among a plurality of displayed images.For example, as shown in FIGS. 10A and 10B, images 1010, 1020, and 1030where event information is matched and stored may be displayed togetherwith the icons 1011, 1021, and 1031 for identifying whether eventinformation is matched.

Moreover, the icons 1011, 1021, and 1031 for identifying whether eventinformation is matched may include activation icons 1011 and 1021 and adeactivation icon 1031.

Herein, among the images 1010, 1020, and 1030 where event information ismatched, the activation icons 1011 and 1021 are icons for identifyingthe images 1010 and 1020 displayed when an event occurs and among theimages 1010, 1020, and 1030 where event information is matched, thedeactivation icon 1032 is an icon for identifying the image 1030 notdisplayed even when an event occurs.

For example, when event information “A region” is matched to the firstimage 1010, the control unit 180 may display the first image 1010 whenthe mobile terminal 100 is located in the “A region”. For anotherexample, when event information “B time point” is matched to the thirdimage 1030, the control unit 180 may not display the third image 1030when the “B time point” arrives.

Moreover, the activation icons 1011 and 1021 and the deactivation icon1031 may be displayed with different colors.

Whether to activate the display when an event occurs may be setaccording to a user input.

For example, when a user input for the first activation icon 1011 isreceived, the control unit 180 may change the activation icon 1011 intoa deactivation icon and display it and even when an event correspondingto the first image 1010 occurs, may not display the first image 1010.

For another example, when a user input for the third deactivation icon1031 is received, the control unit 180 may change the deactivation icon1031 into an activation icon and display it and even when an eventcorresponding to the third image 1030 occurs, display the third image1030.

Moreover, a plurality of displayed images may include images 1040, 1050,and 1060 where event information is not matched. In this case, when auser input for inputting event information to one of the images 1040,1050, and 1060 where event information is not matched is received, thecontrol unit 180 may match event information inputted by a user to animage selected by a user and store it in the storage unit 170. That is,the control unit 180 may store the image matched to the inputted eventinformation by matching the inputted event information to a pre-storedimage and storing the inputted event information. Herein, the eventinformation inputted by a user may include at least one of informationon a time point that an image selected by a user is displayed,information on a place where an image selected by a user is displayed,and information on a specific application to be displayed together withan image an image selected by a user when an application is executed.

For example, as shown in FIG. 10B, after the fourth image 1040 isselected by a user input, if a user input for a time point that thefourth image 1040 is displayed is received, the control unit 180 maymatch the fourth image 1040 to event information on the time point thatthe fourth image 1040 is displayed and store the fourth image 1040matched to the event information on the time point that the fourth image1040 is displayed in the storage unit 170.

In such a way, by allowing a user to input event information to anobtained image, the obtained image may be managed as an image forproviding information even if it is not an image captured through aspecific capturing mode.

FIGS. 11A and 11B are diagrams illustrating examples of obtaining eventinformation on an image captured through a specific capturing mode andmatching it to an image according to some implementations.

The control unit 180 may obtain event information on an image capturedthrough a specific capturing mode and match the captured image to theevent information and store the image matched the event information.

Herein, the event information may be obtained based on at least one ofan object included in an image captured through a specific capturingmode, a text included in an image captured through a specific capturingmode, information on a time point that an image is captured through thespecific capturing mode, and information on an application executed whenan image is captured through the specific capturing mode.

For example, as shown in FIG. 11A, the control unit 180 may obtaininformation on a place where a picture is taken based on objects 1111and 1112 included in an image 1110 captured through a specific capturingmode. That is, since the object representing a parking zone sign and theobject 1112 representing a vehicle are included in the image 1110captured through a specific capturing mode, the control unit 180 maydetermine that the image 1110 is taken at a parking lot.

Then, the control unit 180 may match information on a place where theimage 1110 is captured through a specific capturing mode, as eventinformation, to the image 1110 captured through the specific capturingmode and store it. For example, the control unit 180 may match placeinformation “parking lot” as event information to the image 1110captured through a specific capturing mode and store it. That is, thecontrol unit 180 may store the image 1110 matched to the eventinformation (parking lot) by matching the event information (parkinglot) to the image 1110 and storing the image 1110 and the eventinformation (parking lot).

Moreover, the location information “parking lot” may be obtained byusing context information. For example, the control unit 180 may learnlocation information of a parking lot (for example, the coordinates of aparking lot or wireless signals detected from a parking lot) on thebasis of coordinates when a user visits the “parking lot” and wirelesssignals when a user visits the “parking lot”.

Then, when the mobile terminal 100 moves to the location of a parkinglot obtained by using context information, the control unit 180 maydisplay the image 1110 where location information “parking lot” ismatched as event information.

Additionally, the control unit 180 may match the location information ofa mobile terminal (for example the coordinates of a parking lot orwireless signals detected from a parking lot) when the image 1110 iscaptured, as event information, to the image 1110 and store it.

Then, when the mobile terminal 100 moves to the location of a mobileterminal where the image 1110 is captured, the control unit 180 maydisplay the image 1110 where the location information of the mobileterminal is matched as event information.

Moreover, as shown in FIG. 11B, the control unit 180 may obtain at leastone of information on a place and time information that an image is tobe displayed based on the texts 1131 and 1132 included in the image 1130captured through a specific capturing mode.

For example, the text 1131 representing a time and the text 1132representing a place are included in the image 1130 captured through aspecific capturing mode.

Then, the control unit 180 may obtain event information on a time pointthat the image 1130 is displayed on the basis of the text 1131representing a time in the image 1130. For example, as shown in FIG.11B, when Hong Gildong's lecture is held June 11, the control unit 180may display the image 1130 when it comes to June 11.

Additionally, the control unit 180 may obtain event information on aplace where the image 1130 is displayed on the basis of the text 1132representing a place in the image 1130. For example, as shown in FIG.11B, when Hong Gildong's lecture is held in a “D university,” thecontrol unit 180 may allow a user to check the details of the lecture bydisplaying the image 1130 when a mobile terminal moves to the “Duniversity.”

Additionally, the control unit 180 may match event information obtainedfrom the image 1130 to the image 1130 and store it.

Moreover, the location information of the “D university” may be obtainedby using context information. For example, the control unit 180 maylearn location information of the “D university” (for example, thecoordinates of the D university or wireless signals detected from the Duniversity) on the basis of at least one of the coordinates when a uservisits the “D university” and wireless signals when a user visits the “Duniversity”.

Additionally, the location information of the “D university” may beobtained through wired/wireless network. For example, the control unit180 may obtain the location information of the “D university” byobtaining the location information of the “D university” on internetthrough wired/wireless network.

Then, when the mobile terminal 100 moves to the location of the “Duniversity”, the control unit 180 may display the image 1130 where thelocation information of the “D university” is matched as eventinformation.

Then, when it comes to “June 11”, the control unit 180 may display theimage 1130 where time point information “June 11” is matched as eventinformation.

Moreover, the control unit 180 may match information on a time pointthat an image is captured through a specific capturing mode, as eventinformation, to the image captured through the specific capturing modeand store it. For example, when an image is captured at 03:00 pm Aug.15, 2015, the control unit 180 may match time point information “03:00pm Aug. 15, 2015” as event information to an image.

Then, the control unit 180 may display an image at a time pointcorresponding to the time point information matched as eventinformation. For example, the control unit 180 may display an imagewhere the time point information “03:00 pm Aug. 15, 2015” is matched asevent information every day. For another example, the control unit 180may display an image where the time point information “03:00 pm Aug. 15,2015” is matched as event information, at 03:00 pm Aug. 15, 2016.

Moreover, the control unit 180 may match information on an applicationexecuted at a time point that an image is captured through a specificcapturing mode, as event information, to the image captured through thespecific capturing mode and store it. For example, in a state that anexecution screen of an “A application” is displayed on the screen of themobile terminal 100 as the “A application” is executed, when a “B image”is captured through a specific capturing mode, the control unit 180 maymatch the “A application” as event information to the “B image” andstore it.

Then, in the case that specific application information is matched to aspecific image, when a specific application is executed, the controlunit 180 may display a specific image where the specific applicationinformation is matched. For example, when the “A application” isexecuted, the control unit 180 may display the “B image” together withthe execution screen of the “A application.”

In such a way, unlike the implementations of FIGS. 6A to 10B in whichevent information is matched to an image by a user input, according tothis implementation, event information is automatically matched to animage on the basis of a captured image, location information duringcapturing, time point information during capturing, and information onan application executed during capturing, so that a user's conveniencemay be improved.

FIGS. 12 to 15 are diagrams illustrating a process of classifying aplurality of images obtained by a mobile terminal and displaying them byeach category according to some implementations.

FIG. 12 is a view illustrating a plurality of images 1205, 1210, 1215,1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 obtained by themobile terminal 100. The plurality of images shown in FIG. 12 are imagesobtained through a specific capturing mode but implementations are notlimited thereto.

The control unit 180 may determine a category of each of the pluralityof obtained images 1205, 1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245,1250, 1255, and 1260, and display some of the plurality of obtainedimages 1205, 1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255,and 1260 by each category. In more detail, the control unit 180 maydetermine a category of each of the plurality of obtained images 1205,1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 onthe basis of at least one of an object detected in the image andinformation on a place where a plurality of images are obtained, whichare included in each of the plurality of obtained images 1205, 1210,1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260, anddisplay some of the plurality of obtained images 1205, 1210, 1215, 1220,1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 by each category.

FIGS. 13A and 13B are diagrams illustrating examples of displaying aplurality of images by each category according to some implementations.

When an image is captured, the control unit 180, as mentioned above, mayobtain location information of the captured image. In more detail, thecontrol unit 180 may obtain information on a location where an image iscaptured on the basis of, for example, GPS coordinates, wirelesssignals, or context information during image capturing, etc.

Additionally, the control unit 180 may classify a plurality of images(e.g., the plurality of images 1205, 1210, 1215, 1220, 1225, 1230, 1235,1240, 1245, 1250, 1255, and 1260 in FIG. 12) by each category anddisplay UI objects 1310, 1320, and 1330 for displaying an image by eachcategory according to a user selection. For example, the control unit180 may classify, as a first category, a subset of images among theplurality of images (e.g., a subset of the plurality of images 1205,1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 inFIG. 12) that were obtained at a first location. The control unit 180may then display a first UI object 1310 for displaying the subset ofimages classified as the first category according to a user selection.

Additionally, as shown in FIG. 13A, when a category is classifiedaccording to locations where the plurality of images 1205, 1210, 1215,1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 are obtained,the control unit 180 may display information regarding a location wherean image is obtained together with the UI objects 1310, 1320, and 1330for displaying an image by each category. For example, when a locationwhere images classified as the first category is determined to be an “Amart” location, then the “A mart” icon may be displayed together on thefirst UI object 1310.

Moreover, when the first UI object 1310 is selected from the pluralityof displayed UI objects 1310, 1320, and 1330, the control unit 180 maydisplay images classified as the first category, that is, a subset ofimages 1205, 1215, 1235, 1245, 1260 obtained at the first location.

In such a way, pictures that a user captures to obtain information areclassified by a location and displayed, so that user's informationacquisition may be accomplished easily. For example, when a user shopsat an A mart and takes pictures of the types, designs, prices ofproducts sold in the A mart to remember them, according to someimplementations, the pictures captured at the A mart are classified anddisplayed, so that user's information acquisition may become easy.

FIGS. 14A to 14C are diagrams illustrating examples of displaying aplurality of images depending on an object detected in an image,according to some implementations.

The control unit 180 may extract a feature vector from a plurality ofimages (e.g., the plurality of images 1205, 1210, 1215, 1220, 1225,1230, 1235, 1240, 1245, 1250, 1255, and 1260 in FIG. 12), and comparethe extracted feature vector with a feature vector database, so that itmay determine which object is included in each of the plurality ofimages 1205, 1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255,and 1260. For example, the control unit 180 may determine that an objectincluded in each image of a subset of images 1205, 1235, 1250, and 1255is a “price tag” by using a feature vector extracted from the images1205, 1235, 1250, and 1255 of FIG. 12. For another example, the controlunit 180 may determine that an object included in each of the images1215 and 1245 is a “bookshelf” by using a feature vector extracted fromthe images 1215 and 1245 of FIG. 12.

As shown in FIG. 14A, the control unit 180 may classify the plurality ofimages (e.g., the plurality of images 1205, 1210, 1215, 1220, 1225,1230, 1235, 1240, 1245, 1250, 1255, and 1260 in FIG. 12) by eachcategory and display UI objects 1410, 1420, and 1430 for displaying animage by each category according to a user selection. For example, thecontrol unit 180 may classify, as a first category, images including afirst object among the plurality of images 1205, 1210, 1215, 1220, 1225,1230, 1235, 1240, 1245, 1250, 1255, and 1260 and display a first UIobject 1410 for displaying images classified as the first categoryaccording to a user selection. For another example, the control unit 180may classify, as a second category, images including a second objectamong the plurality of images 1205, 1210, 1215, 1220, 1225, 1230, 1235,1240, 1245, 1250, 1255, and 1260 and display a second UI object 1420 fordisplaying images classified as the second category according to a userselection.

Moreover, when a specific UI object 1410 is selected from the pluralityof displayed UI objects 1410, 1420, and 1430, the control unit 180 maydisplay at least a part of an image corresponding to a UI objectselected by a user.

For example, as shown in FIG. 14B, when the first UI object 1410 isselected from the plurality of displayed UI objects 1410, 1420, and1430, the control unit 180 may display images classified as the firstcategory, that is, at least part of the images 1205, 1235, 1250, and1255 including the first objects 1206, 1235, 1251, and 1256.

For another example, as shown in FIG. 14C, when the second UI object1420 is selected from the plurality of displayed UI objects 1410, 1420,and 1430, the control unit 180 may display images classified as thesecond category, that is, at least part of the images 1215 and 1245including the second objects 1216 and 1246.

In such a way, pictures that a user captures to obtain information areclassified by an object detected in an image and displayed, so thatuser's information acquisition may be accomplished easily.

For example, when a user takes a picture of a “price tag” in order tocompare prices for the same product while shopping in an A mart and a Bmart, implementations described herein may classify and display the“price tag” pictures, so that user's information acquisition may becomeeasy.

For another example, when a user takes a picture of a “bookshelf” inorder to compare designs for the “bookshelf” while shopping in an A martand a B mart, implementations described herein may classify and displaythe “bookshelf” pictures, so that user's information acquisition maybecome easy.

FIG. 15 is a diagram illustrating a process of displaying a plurality ofimages depending on a place and an object detected in the image,according to some implementations.

When capturing a plurality of images (e.g., images 1205, 1210, 1215,1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 in FIG. 12),the control unit 180 may obtain place information of the capturedimages. Additionally, the control unit 180 may extract a feature vectorfrom a plurality of images 1205, 1210, 1215, 1220, 1225, 1230, 1235,1240, 1245, 1250, 1255, and 1260, and compare the extracted featurevector with a feature vector database, so that it may determine whichobject is included in each of the plurality of images 1205, 1210, 1215,1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260.

Then, the control unit 180 may display the plurality of images 1205,1210, 1215, 1220, 1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 byeach category on the basis of a place where an image is captured and anobject included in an image. For example, the control unit 180 mayclassify a category of the plurality of images 1205, 1210, 1215, 1220,1225, 1230, 1235, 1240, 1245, 1250, 1255, and 1260 according to whetheran image includes an object that is common to each other and is obtainedat the same place. Then, the control unit 180 may display a selectedsubset of the plurality of images, such as images 1205 and 1235 in FIG.15, that have an object common to each other and that were obtained atthe same place.

In such a way, pictures that a user captures to obtain information areclassified by an object detected in an image and displayed, so thatuser's information acquisition may be accomplished easily.

For example, when a user takes pictures of the price tags of bookshelvesin order to compare the prices of the bookshelves while shopping in an“A mart” and takes pictures of beds in order to compare the designs ofthe beds while shopping in the “A mart” and takes pictures of the pricetags of chairs in order to compare the prices of the chairs whileshopping in a “B mart,” implementations described herein may classifyand display the price tag images of the bookshelves captured in the “Amart,” so that user's information acquisition may become easy.

FIGS. 16A and 16B are diagrams illustrating a process of classifying anddisplaying an image including text among a plurality of images accordingto an implementation.

Referring to FIG. 16A, when images 1610, 1620, 1630, and 1640 arecaptured through a specific capturing mode, the control unit 180 mayrecognize texts included in the images 1610, 1620, 1630, and 1640captured through the specific capturing mode and classify the images1610, 1620, 1630, and 1640 including the texts.

Then, the control unit 180 may display a folder including the images1610, 1620, 1630, and 1640 including the texts, and when the folder isselected by a user, as shown in FIG. 16B, the control unit 180 maydisplay the images 1610, 1620, 1630, and 1640 including the texts.

Moreover, this implementation describes that images are classified anddisplayed based on whether text is included, but implementations are notlimited thereto.

In more detail, the control unit 180 may classify and display images onthe basis of texts and objects detected in an image captured through aspecific capturing mode. As an example, if the images shown in FIGS. 16Aand 16B were captured through a specific capturing mode from writtencontents on a white board, such that a user divides and captures textswritten on a white board, then the control unit 180 may classify anddisplay images including the white board and the texts among the imagescaptured through the specific capturing mode.

Additionally, the control unit 180 may classify and display images onthe basis of texts included in an image captured through a specificcapturing mode based on a location where an image is captured. Forexample, when a user captures texts at an “A institute” location andcaptures texts at a “B institute” location through a specific capturingmode in order to remember the lectures, then the control unit 180 mayclassify and display images including the texts according to whether theimages were captured at the “A institute” or the “B institute.”

Referring back to FIG. 3, in the example process of operating a mobileterminal according to an implementation, if a specific event occurs,then an image is displayed corresponding to the specific event among atleast one image that is matched to each of at least one eventinformation and stored in operation S350. In more detail, when aspecific event occurs, the control unit 180 may obtain event occurrenceinformation on each of a plurality of images by using a plurality ofevent information matched to the plurality of images and display animage corresponding to the specific event among the plurality of images.This will be described in more detail with reference to FIGS. 17A to 17Dand FIG. 19, below.

FIGS. 17A to 17D are diagrams illustrating a process of displaying animage when information on a place where the image is displayed ismatched as event information according to some implementations.

When an event occurs indicating that the mobile terminal 100 moves to aspecific place, the control unit 180 may obtain event occurrenceinformation for each of a plurality of images by using event informationmatched to the plurality of images. For example, when the mobileterminal 100 moves to a parking lot, the control unit 180 may obtain thelocation information of the mobile terminal 100. Location informationmay be obtained by any suitable technique, for example, on the basis ofthe GPS coordinates of the parking lot or wireless signal informationsuch as Bluetooth signals transmitted from an electronic device in avehicle parked in the parking lot, BLE signals transmitted from Beacon,WiFi signals transmitted from a wireless Access Point (AP), and/orsignals transmitted during an NFC process. If the control unit 180determines that the current location information of the mobile terminal100 corresponds to location information matched to the third image 1770among the plurality of images 1710, 1720, 1730, 1740, 1750, 1760, 1770,and 1780 stored in the mobile terminal 100 shown in FIG. 17A, then thecontrol unit 180 may display the third image 1770 that is an imagecorresponding to the currently occurring event.

In some implementations, as shown in FIG. 17B, the control unit 180 maydisplay a portion 1771 of the third image 1770 on the screen edge of themobile terminal 100. For example, as shown in FIG. 17B, a portion 1771of a first area of the third image 1770 is displayed on the screen and aportion 1772 of a second area of the third image 1770 is not displayedon the screen. In some implementations, the control unit 180 may controlthe display unit 151 to display the portion 1771 of the first area ofthe third image 1770 along the screen edge.

Furthermore, although this example describes a portion of the thirdimage 1770 displayed along the screen edge, implementations are notlimited thereto. In some implementations, the control unit 180 maydisplay all or part of the third image 1770 in another area of thescreen. For example, the control unit 180 may display the entire thirdimage 1770 along the screen edge. As another example, the control unit180 may display the entire third image 1770 in a specific area of thescreen. As another example, the control unit 180 may display the entirethird image 1770 in full screen mode.

In some implementations, as shown in FIG. 17B, when a user input fortouching the portion 1771 of the third image 1770 displayed on thescreen is received, the control unit 180 may control the display unit151 to display the entire third image 1770 on the screen of a mobileterminal. In this case, the user input may be, as an example, a userinput for touching and dragging the portion 1771 of the third image1770.

FIG. 17C is a diagram illustrating the third image 1770 displayed on ascreen. Although FIG. 17C illustrates that the third image 1770 isdisplayed in full screen mode, implementations are not limited theretoand the third image 1770 may be displayed in a specific area of thescreen of the mobile terminal 100.

In some implementations, when displaying the third image 1770, thecontrol unit 180 may display a menu 1790 for receiving a user input fordeleting the third image 1770, and/or a menu 1795 for receiving a userinput for saving the third image 1770 in computer memory (e.g., storageunit 170).

In some implementations, when a user touch input on the menu 1790 fordeleting the third image 1770 is received, the control unit 180 maydelete the third image 1770 from the storage unit 170 and as shown inFIG. 17D, may display a UI object 1796 for notifying the user that thethird image 1770 is deleted.

Additionally, when a user touch input on the menu 1795 for storing thethird image 1770 is received, the control unit 180 may maintain thethird image 1770 stored in the storage unit 170.

In such a way, an image that a user captures to obtain information or animage captured through a general capturing mode but classified as animage for obtaining information by inputting event information may beprovided in a place where a user seeks information.

Additionally, since only a portion of an image for obtaining informationmay be displayed at the edge of the screen without blocking the entirescreen, such implementations may improve user convenience and reducedistractions when a user uses a mobile terminal.

Additionally, by enabling a user to obtain information by using an imagethat corresponds to that information, implementations described hereinmay enable easily deleting or replacing an image that loses relevance tothat information.

FIGS. 18A and 18B are diagrams illustrating a process of displaying animage when place information stored as event information is wirelesssignal information.

Referring to the example of FIG. 18A, a plurality of images 1810, 1820,1830, 1840, 1850, 1860, 1870, and 1880 are stored in the mobile terminal100. If one of the images is accessed while the mobile terminal 100 isconnected to a wireless communication network, then information aboutthe wireless signal may be stored in the event information that ismatched to that image. As an example, a user may input event informationsuch as “display the eighth image again when arriving at a B conveniencestore” with respect to the eighth image 1880. Upon arriving at the “Bconvenience store,” if the mobile terminal 100 detects a wirelesssignal, then the wireless signal information at the B convenience storemay be stored as event information that is matched to the eighth image1880.

Subsequently, as the mobile terminal 100 moves between differentlocations, if the mobile terminal 100 detects the particular wirelesssignal information corresponding to the B convenience store that isstored event information for the eighth image 1880, then the controlunit 180 may automatically display the eighth image 1880 correspondingto the occurring event. In this case, the control unit 180, as shown inFIG. 18B, may display a portion 1881 of the eighth image 1880 at thescreen edge.

In such a way, implementations described herein enable matching eventinformation, such as wireless signal information, to an image. The eventinformation may correspond to a single location or may correspond tomultiple locations that share a common characteristic. For example, whena user matches an “A convenience store” as event information to an “Aconvenience store discount coupon image,” then the “A convenience storediscount coupon image” may be displayed in a plurality of “A conveniencestores” located in a plurality of locations.

FIG. 19 is a diagram illustrating a process of displaying an image whenapplication information to be displayed with the image is matched to theimage.

A plurality of images stored in the mobile terminal 100 may includeimages where information on a specific application to be displayed withan image during application execution is matched as event information.

Then, as a user input for executing a specific application is received,the control unit 180 may display an execution screen 1919 of thespecific application. Then, the control unit 180 may display a portion1910 of an image where specific application information is matched asevent information when a specific application is executed.

For example, when event information of an “A bank electronic paymentapplication” is matched to a “security card” image and stored, when the“A bank electronic payment application” is executed, the control unit180 may display the “security card” image together with the executionscreen of the “A bank electronic payment application”.

In such a way, in a case that specific information is required when auser executes a specific application, by displaying an imagecorresponding to the specific application, information that a userrequires may be provided automatically when an application is executed.

Moreover, the control unit 180 may display an image where specificapplication information as event information is matched after anauthentication on application execution is completed. In more detail,when a specific application is executed and a user authentication forthe executed application is required, the control unit 180 may displayan image where specific application information as event information ismatched together with an application screen after the userauthentication is completed. Therefore, an image for requiring securitysuch as an electronic payment security card may be displayed after userauthentication is completed.

The above example describes that when place information is matched asevent information, application information is matched as eventinformation. However, implementations are not limited thereto. Forexample, as shown in FIGS. 8A to 8E, in a case that information on atime point that an image is displayed is matched to the image, when anevent reaching a specific time point occurs, the control unit 180 maydisplay an image where event information corresponding to the occurringevent is matched. For example, as the user input “display a first imageon Nov. 26, 2015” is received, when the time point information “Nov. 26,2015” is matched as event information, the control unit 180 may controlthe display unit 151 to display the first image on Nov. 26, 2015.

Additionally, the control unit 180 may determine whether an event occurson the basis of a user input and additional information in a mobileterminal. In relation to this, it will be described in detail withreference to FIGS. 20A to 20C.

FIGS. 20A to 20C are diagrams illustrating a process of matching eventinformation and determining whether an event occurs on the basis ofschedule information stored in a mobile terminal and a user inputaccording to some implementations.

The control unit 180 may match event information to a specific image andstore it on the basis of a user input and schedule information in amobile terminal. For example, as shown in FIG. 20A, when a user inputfor displaying a specific image 2010 on a specific person's birthday isreceived, the control unit 180 may obtain time information 2020 on thespecific person's birthday from schedule information stored in thestorage unit 170 and match the obtained time point information as eventinformation to the specific image 2010 and store it.

If a user input for displaying an image again on a Hong Gildong'sbirthday is received, the control unit 180 may obtain information thatthe Hong Gildong's birthday is Oct. 11, 2015 from schedule informationstored in the storage unit 170 and match the obtained time pointinformation as event information to the specific image 2010 and storeit.

Additionally, on Oct. 11, 2015, the control unit 180, as shown in FIG.20B, may display a portion 2011 of an image where event information Oct.11, 2015 is matched.

FIG. 20C is a view illustrating that an image 2010 where eventinformation is matched is displayed on a screen as a user touch input onthe portion 2011 of the image shown in FIG. 20B is received.

When an event occurs, the control unit 180 may transmit an imagecorresponding to the occurring event to a mobile terminal of a specificperson relating to the occurring event on the basis of at least one of auser input and schedule information in the mobile terminal 100.

For example, when event information “Oct. 11, 2015” is matched to aspecific image and an event that it comes to Oct. 11, 2015 occurs, thecontrol unit 180 may transmit an image where a Hong Gildong's birthdayis matched as event information to Hong Gildong on the basis of scheduleinformation that the Hong Gildong's birthday is Oct. 11, 2015 and HongGildong's phone number information stored in the storage unit 170.

The above example describes the time point information “Oct. 11, 2015”being matched as event information. However, implementations are notlimited thereto. For example, the control unit 180 may match time pointinformation “Hong Gildong's birthday” as event information.

Then, in a case that event information “Hong Gildong's birthday” ismatched to a specific image, when an event that it comes to Oct. 11,2015 occurs, the control unit 180 may obtain event occurrenceinformation on a specific image on the basis of the event information“Hong Gildong's birthday” and additional information in the storage unit170, and display a specific image corresponding to the occurring event.

Additionally, on the basis of Hong Gildong's phone number informationstored in the storage unit 170, an image corresponding to the occurringevent may be transmitted to a Hong Gildong's mobile terminal.

FIG. 21 is a diagram illustrating a process of displaying additionalinformation on an image displayed when an event occurs.

Referring to FIG. 21, as a specific event occurs, when a user input forobtaining additional information is received in a state that an imagecorresponding to the specific event is displayed, the control unit 180may obtain additional information′ 2130 relating to the displayed imageand display it.

In detail, as a user input for touching a portion 2111 of an imagecorresponding to an occurring event is received, the control unit 180may display an image 2110 corresponding to the occurring event.

Then, as a user input for dragging the displayed image 2110 is received,the control unit 180 may display a UI object 2120 for notifying thatadditional information relating to the image 2110 can be requested.

Moreover, according to a user input for requesting additionalinformation relating to the image 2110, the control unit 180 may obtainthe additional information relating to the image 2110 and display theobtained additional information 2130.

In more detail, when a user input for obtaining additional informationon the displayed image 2110 is received, the control unit 180 may obtainadditional information on the displayed image 2110 through web searchusing wireless internet and display it.

For example, as shown in FIG. 21, when a user input for searching for animage corresponding to the displayed image 2110 is received, the controlunit 180 may search for an image corresponding to an object detected inthe displayed image 2110 through a search engine and display a searchresult.

For another example, when a user input for identifying a location on amap of the displayed image 2110 is received, the control unit 180 maydisplay an image displaying the location of the displayed image 2110 onthe map.

FIG. 22 is a diagram illustrating a process of displaying additionalinformation on a displayed image according to another implementation.

When a user input for obtaining additional information relating to thedisplayed image 2210 is received, the control unit 180 may obtain theadditional information relating to the image 2210 and display theobtained additional information 2230.

For example, as shown in FIG. 22, when a user input for comparing theprice of an object detected in the displayed image 2210 is received, thecontrol unit 180 may search for price information of a product identicalto the object detected in the displayed image 2210 through web searchusing wireless Internet and display a search result.

Moreover, this implementation describes that additional information isobtained through web search, but implementations are not limitedthereto. In more detail, when a user input for obtaining additionalinformation on a displayed image is received, the control unit 180 mayobtain additional information on the displayed image by using datastored in the mobile terminal 100 and display it. For example, thecontrol unit 180 may display an image that is classified as the samecategory as the displayed image. For another example, in a case that animage is displayed as an event that reaches a specific time pointoccurs, the control unit 180 may display schedule informationcorresponding to the occurring event in schedule information pre-storedin the storage unit 170, as additional information.

FIGS. 23A to 23C are diagrams illustrating an example of updating aschedule on the basis of a text included in an image captured through aspecific image capturing mode.

As shown in FIG. 23A, the control unit 180 may obtain event informationon an image captured through a specific capturing mode and match thecaptured image to the event information and store it. Herein, the eventinformation may be obtained based on a text included in an imagecaptured through a specific capturing mode. For example, the controlunit 180 may obtain event information on a time point or a place fordisplaying an image 2210 by recognizing texts 2211 and 2212 included inthe image 2210 captured through a specific capturing mode. The controlunit 180 may then match the obtained event information to the image 2210and store it.

Additionally, the control unit 180, as shown in FIG. 23B, may updateschedule information on the basis of the text included in the image(e.g., texts 2211 and 2212 in the image 2210 of FIG. 23A) capturedthrough a specific capturing mode. In more detail, the control unit 180may obtain time information and location information 2320 from the texts2211 and 2212 included in the image 2210 captured through a specificcapturing mode, and may update schedule information on the basis of theobtained information.

Additionally, when an event occurs, the control unit 180, as shown inFIG. 23C, may control the display unit 151 to display a portion 2311 ofan image corresponding to the occurring event.

FIG. 24 is a diagram illustrating a process of performing textconversion from an image obtained through a specific capturing mode.

The control unit 180 may convert a text image 2411 included in an image2410 obtained through a specific capturing mode into a text. In moredetail, when a user input for converting an image into a text isreceived, the control unit 180 may display a menu 2420 for setting amethod of storing an image. Then, when a user input for a menu forconverting the image 2410 into a text is received, the control unit 180may convert the text image 2411 included in the image 2410 into a text.

An image for providing information may include a text often. Therefore,according to some implementations, as the content of a text included inan image is automatically converted into a text without a user's inputagain, user's convenience may be improved.

Moreover, the control unit 180, as a component responsible forcontrolling a device generally, may be used interchangeably with theterm of a central processing device, a micro-processor, and a processor.

Implementations may also be implemented as computer readable codes on acomputer readable recording medium. The computer readable recordingmedium is any data storage device that can store data which can bethereafter read by a computer system. Examples of the computer readablerecording medium include magnetic storage media (e.g., ROM, floppydisks, hard disks, etc.) and optical recording media (e.g., CD-ROMs, orDVDs) and carrier waves (e.g., transmission through the Internet).Additionally, the computer may include the control unit 180 of aterminal. Accordingly, the detailed description is not construed asbeing limited in all aspects and should be considered as illustrative.Various modifications of the above examples are included in the scope ofthis disclosure.

What is claimed is:
 1. A mobile terminal comprising: a memory; an inputinterface; a display; and a controller configured to: receive a voiceinput to input event information relating to an image through the inputinterface, while the image is displayed; store, in the memory, the eventinformation relating to the image; detect an event; determine that thedetected event corresponds to the event information; and display, on thedisplay, the image related to the event information, based on thedetected event corresponding to the event information, wherein the eventinformation includes at least one of time information indicating a timepoint at which the image is to be displayed or location informationindicating a location where the image is to be displayed, wherein thecontroller is configured to receive the voice input to input the eventinformation by: receiving a first voice input to input first eventinformation relating to a first image, wherein the first eventinformation relating to the first image includes first time informationindicating a first time point at which the first image is to bedisplayed, and wherein the controller is configured to display the imagerelated to the event information by: displaying the first image based onoccurrence of the first time point.
 2. The mobile terminal according toclaim 1, wherein the event information is learned based on at least oneof a user input, location of the mobile terminal, a visit history, awireless signal sensed by the mobile terminal, or information obtainedthrough wired/wireless network.
 3. The mobile terminal according toclaim 1, wherein the controller is further configured to output a queryfor obtaining the time point at which the image is to be displayed orthe location where the image is to be displayed, based on the time pointat which the image is to be displayed or the location where the image isto be displayed not being specified by the voice input.
 4. The mobileterminal according to claim 1, further comprising: a wirelesscommunicator configured to communicate with an external device, whereinthe controller is configured to transmit the displayed image to aspecific person's mobile terminal based on the specific person's phonenumber information stored in the memory.
 5. The mobile terminalaccording to claim 1, wherein the controller is configured to: store aplurality of event information relating, respectively, to a plurality ofimages; and display the plurality of images.
 6. The mobile terminalaccording to claim 1, wherein the controller is configured to obtainadditional information relating to the displayed image through websearch and display the obtained additional information, wherein theadditional information includes at least one of an image or priceinformation corresponding to an object detected in the displayed image.7. The mobile terminal according to claim 6, wherein the objectcomprises a price tag.
 8. The mobile terminal according to claim 1,wherein the controller is further configured to display the obtainedimage at an edge of a screen of the mobile terminal.
 9. The mobileterminal according to claim 1, wherein the event information includesapplication information indicating an application that is to bedisplayed together with the obtained image.
 10. The mobile terminalaccording to claim 1, wherein the controller is configured to controlthe display to display a first portion of the image related to the eventinformation at an edge of a screen of the display, and control thedisplay to display a second portion of the image related to the eventinformation that includes both the first portion and a third portion ofthe image related to the event information different than the firstportion based on a user input to select the displayed first portionbeing received.
 11. The mobile terminal according to claim 10, whereinthe controller is further configured to control the display to display afirst user interface configured to receive a user input to delete thedisplayed image and a second user interface configured to receive a userinput to store the displayed image.
 12. A method of operating a mobileterminal, the method comprising: receiving a voice input to input eventinformation relating to an image, while the image is displayed; storing,in a memory, the event information relating to the image; detecting anevent; determining that the detected event corresponds to the eventinformation; and displaying, on a display, the image related to theevent information, based on the detected event corresponding to theevent information, wherein the event information includes at least oneof time information indicating a time point at which the image is to bedisplayed or location information indicating a location where the imageis to be displayed, wherein the receiving the voice input to input theevent information comprises: receiving a first voice input to inputfirst event information relating to a first image, wherein the firstevent information relating to the first image includes first timeinformation indicating a first time point at which the first image is tobe displayed, and wherein the displaying the image related to the eventinformation comprises: displaying the first image based on occurrence ofthe first time point.
 13. The method according to claim 12, wherein theevent information is learned based on at least one of an user input,location of the mobile terminal, a visit history, a wireless signalsensed by the mobile terminal, or information obtained throughwired/wireless network.
 14. The method according to claim 12, whereinreceiving the voice input comprises outputting a query for obtaining thetime point at which the image is to be displayed or the location wherethe image is to be displayed, based on the time point at which the imageis to be displayed or the location where the image is to be displayednot being specified by the voice input.
 15. The method according toclaim 12, further comprising: storing a plurality of event informationrelating, respectively, to a plurality of images; and displaying theplurality of images.
 16. The method according to claim 12, furthercomprising: obtaining additional information relating to the displayedimage through web search and displaying the obtained additionalinformation, wherein the additional information includes at least one ofan image or price information corresponding to an object detected in thedisplayed image.
 17. The method according to claim 12, whereindisplaying the image comprises: displaying a first portion of the imagerelated to the event information at an edge of a screen of the display,and displaying a second portion of the image related to the eventinformation that includes both the first portion and a third portion ofthe image related to the event information different than the firstportion, based on a user input to select the displayed first portionbeing received.
 18. The method according to claim 17, wherein displayingthe image comprises: displaying a first user interface configured toreceive a user input to delete the displayed image and a second userinterface configured to receive a user input to store the displayedimage.